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Interactive Music Scores 


• Alien Lands - Sandeep Bhagwati 
[Montreal - February 2011] 

Music performance in four movements 
for four spatially dispersed 
percussionists with interactive scores. 


• Calder’s Violin - Richard Hoadley 
[Cambridge - October 2011] 

Automatic music for violin and computer. 










The Interlude Project 

New Digital Paradigms for Exploration and Interaction 

of Expressive Movement with Music. 






The Interlude Project 





INScore 


• Music score extension 

• Graphic & time spaces relationship 

• Performance representation 


• Interaction 




INScore 


INScore supports 

• Symbolic music notation [ GMN, MusicXML] 

• Textual elements 

• Bitmaps [jpg, gif, tiff, png,...] 

• Vectorial graphics (rectangles, ellipses, SVG,...) 

• Video files 

• Sound and gesture graphic representations 



INScore is 


• a standalone score viewer 

• an open source C / C++ library 

• multi-platform 

• an Open Sound Control API 
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INScore 
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Relations between graphic and time space 


Hypothesis 

Approach the problem with segmentation and relations between segments 
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Relations between graphic and time space 


Hypothesis 

Approach the problem with segmentation and relations between segments 


Segments 

Defined as a list of intervals: 

• generalizable to n dimensions 

• intersection operation 


Segmentation 

A set of disjoined segments 
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Relations between graphic and time space 


Hypothesis 

Approach the problem with segmentation and relations between segments 




Relations between graphic and time space 


Hypothesis 

Approach the problem with segmentation and relations between segments 


Mapping 

Relation between two segmentations: 

• operations to query the mapping 

• operations to compose mappings 




Relations between graphic and time space 


Segmentations and mappings for each component type: 


type 

segmentations and mappings required 

text 

graphic ** text ** relative time 

score 

graphic** wrapped relative time ** relative time 

image 

graphic ** pixel <-> relative time 

vect. graphics 

vectorial relative time 

signal 

graphic ** frame ** relative time 
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Performance representation 


The VEMUS approach 


• a mirror metaphor 

• feedback based pedagogy 

• score annotation with 
performance representations 
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Performance representation 


The VEMUS approach 


• a mirror metaphor 

• feedback based pedagogy 

• score annotation with 
performance representations 

• static design, 

• tricky to extend, 

• awkward to experiment. 



n 







Performance representation 


Hypothesis 

Approach the graphic of a signal as a graphic signal. 
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Performance representation 


Hypothesis 

Approach the graphic of a signal as a graphic signal. 


A graphic signal 

A composite signal made of: 

• a y signal 

• a thickness signal 

• a color signal 



t 
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Performance representation 


Hypothesis 

Approach the graphic of a signal as a graphic signal. 


A graphic signal 

A composite signal made of: 

• a y signal 

• a thickness signal 

• a color signal 



t 


Consider a signal S defined as a time function: f{t ) : R —» R 3 = (y, h, c) \ y, h, C G R 
This signal could be directly drawn (i.e. without additional computation) 
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Performance representation 


Hypothesis 

Approach the graphic of a signal as a graphic signal. 


A graphic signal 

A composite signal made of: 

• a y signal 

• a thickness signal 

• a color signal 

Consider a signal S defined as a time function: f(t ) : R — > R 3 = (y, h, c) \ y, h, C G R 
This signal could be directly drawn (i.e. without additional computation) 
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Performance representation 

System expressivity 


Examples 



g = S f0 / k t / k c 

Sfo : fundamental frequency 
k t : constant thickness signal 
k c : constant color signal 
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Performance representation 

System expressivity 


Examples 



g - ky / Srms / kc 

Srms '■ RMS signal 
k y : constant y signal 
k c : constant color signal 
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Performance representation 

System expressivity 


Examples 



9 — SfQ / Srms / k c 
Srms '■ RMS signal 
Sfo : fundamental frequency 
k c : constant color signal 
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Performance representation 

System expressivity 


Examples 



go — S f0 j Sfmso / k c 0 
Sm : fundamental frequency 
Srmso ■ fO RMS values 

= S f0 / Srms'\ I S r msO / k c ~\ 

Srms i : fl RMS values 

02 = / Srms2 + Srms] ' S rnlS Q / /C c 2 

Srms2 ■ f2 RMS values 

■ ■ ■ 

g = g2/ g J [ / gO 
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INS core OSC Messages 


An «object oriented» approach 
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INS core OSC Messages 


An «object oriented» approach 

• The OSC address is like an object pointer. 

• An OSC message is similar to an object method call. 

• The OSC address space is dynamic. 
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INS core OSC Messages 


An «object oriented» approach 

• The OSC address is like an object pointer. 

• An OSC message is similar to an object method call. 

• The OSC address space is dynamic. 


OSC message general format 



/ITL/scene/score color 255 128 40 150 
score->color(255, 128, 40, 150) 
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INScore OSC Address Space 


OSC address 


message string 
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INScore OSC Address Space 


OSC address 


message string 




Application 


/ scene 1 


/scene2 


Iscene3 
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INScore OSC Address Space 


OSC address 


message string 


parameter 



Application 


Scene 



/signal 

/objectl 


/object2 

/object3 
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INScore OSC Address Space 


OSC address 


message string 




Application 


Scene 


Components 
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INScore OSC Address Space 


OSC address 


message string 




Application 


Scene 


Components 


Signals 
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Messages Strings 




message string 




parameter 


/ 
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Messages Strings 


-1 OSC address I- 

Graphic space control 


message string 



• Position: 

x, y, z, angle, scale, dx, dy, dz, 
dangle, dscale 

• Color: 

color, dcolor, red, green, blue, 
dred, dgreen, dblue, alpha, dalpha, 
hue, saturation, brightness, dhue, 
dsaturation, dbrightness 

Time space control 

• Time position: 

date, ddate, clock 

• Duration: 

duration, dduration 


Constructor 

set 

Query message 

get 

Time and graphic 
spaces relations 

map 

Signals and graphic 
signals messages 
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Messages Parameters 





Direct use of basic OSC types 

• int32 

• float32 

• OSC-string 

Relaxed types but strict parameters count 
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Interaction Messages 


Basic principle 


OSC address 


watch 


event 


message 
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Interaction Messages 


Basic principle 


OSC address 


watch 




• mouse up, mouse down, 
mouse move, mouse enter, 
mouse leave... 

• time enter, time leave 
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Interaction Messages 


Basic principle 



watch 
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Interaction Messages 


Basic principle 















Interaction Messages 


Basic principle 















Interaction Messages 


Basic principle 



watch 




Examples 

/ITL/scene/myObject watch mouseDown ( /ITL/ scene/myObject show 0 


▼ 
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Interaction Messages 


Basic principle 



watch 




Examples 

/ITL/scene/myObject watch mouseDown ( /ITL/ scene/myObject show 0 


▼ 


/ITL/scene/myObject watch mouseDown (host.domain.org: 12100/an/address star-t^ 
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Interaction Messages 


Variables 

$x, $y, $absx, $absy, $sx, $sy 
$date 

Address variables 

$self 

$scene 


Scaling variable values 

$x[min, max], $y[min, max] 

Date quantification 

$date[n/d] 


Message based variables 

• $(a valid INScore 'get' message) 
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Interaction Messages 


Variables 


Scaling variable values 


• $x[min, 

• $x, $y, $absx, $absy, $sx, $sy 

• $date 


Address varial 

• $self 

• $scene 


Message basec 

• $(a valid INScor 


-i 



max], $y[min, 

uantification 

/d] 


max ] 


i 
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Interaction Messages 


Variables 

$x, $y, $absx, $absy, $sx, $sy 
$date 

Address variables 
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$scene 


Scaling variable values 

$x[min, max], $y[min, max] 

Date quantification 

$date[n/d] 


Message based variables 

• $(a valid INScore 'get' message) 
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Scripting 


• INScore files as script files. 

• Supports variables 

• Javascript support (embedded by default) 

<?javascript ... any javascript code ... ?> 

• optional Lua support (not embedded by default) 

<?lua ... any lua code ... ?> 
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http://inscore.sourceforge.net 


open source 





